Repository navigation
docs(rfc): multi-node devbox with GPU/RDMA-capable AppEnvironments - #8218
Open
cosmicbboy wants to merge 1 commit into
Open
cosmicbboy wants to merge 1 commit into
cosmicbboy wants to merge 1 commit into
Conversation
|
This PR was flagged by our automated quality checks. If you're a genuine We appreciate your contribution and apologize if this is a false positive! |
RFC-style spec (rfc/system) proposing: - multi-node devbox: node 2 joins as a k3s agent over the RoCE fabric - AppEnvironment GPU plumbing: map Resources_GPU to nvidia.com/gpu (protoResourceName currently drops it), reuse flytek8s GPU/shared-memory helpers on the app path, auto-label GB10 nodes - RDMA (/dev/infiniband), host-network and memlock/NCCL notes, multi-host TP=2 serving pattern, shared weight storage - live-verified baseline evidence on a dual DGX Spark pair + minimal-change path (PodTemplate apps work today) Signed-off-by: Niels Bantilan <cosmicBboy@users.noreply.github.com>
cosmicbboy
force-pushed
the
docs/devbox-dual-dgx-gpu-rfc
branch
from
October 22, 2026 06:34
cc753e1 to
71bc806
Compare
cosmicbboy
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8218 +/- ##
==========================================
+ Coverage 56.56% 57.34% +0.78%
==========================================
Files 772 931 +159
Lines 52305 58336 +6031
==========================================
+ Hits 29587 33454 +3867
- Misses 19961 21831 +1870
- Partials 2757 3051 +294
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Member
|
@cosmicbboy I think you have the wrong target branch |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Flyte's devbox is architecturally a single-node k3s cluster, and the app-serving path
cannot hand an AppEnvironment a usable GPU today:
pair is invisible to it.
app/internal/k8s/app_client.go::protoResourceNamemaps only CPU/MEMORY/STORAGE/EPHEMERAL_STORAGE to k8s resources —
Resources_GPUis silently dropped, so anAppEnvironment's
gpu=never becomesnvidia.com/gpuon the Knative pod.nvidia) containerd runtime stays in"passthrough when no GPU is requested" mode: pods see the driver interface
(
/proc/driver/nvidia/gpus) but no/dev/nvidia*, no CUDA, and never/dev/infiniband—so cross-node NCCL/RoCE workloads cannot run from an app pod.
This is a design spec (no code change) for making devbox able to span two DGX Sparks and
give GPU/RDMA-capable AppEnvironments, motivated by hosting a TP=2 vLLM DeepSeek service in
Flyte instead of a systemd + docker-compose stack.
What changes were proposed in this pull request?
Adds
rfc/system/20261022-devbox-dual-dgx-gpu.md, an RFC-style spec containing:nvidia.com/gpu: 1, AppEnvironment GPU request dropped to{cpu,memory}, pod sees node-1GB10 driver UUID but no device nodes / Inferfaband).
flyte-sdkCLI agent/join + devbox-bundled entrypoint +registry reachability from node 2).
Resources_GPU→nvidia.com/gpu, call the task path'sflytek8s.SanitizeGPUResourceRequirements/ApplyGPUNodeSelectors, applyExtendedResources.SharedMemoryto a/dev/shmvolume, and auto-label nodes./dev/infiniband), host network, memlock/NCCL-channel notes.multi-rank app later).
protoResourceName), acceptancecriteria, risks/open questions, and a change inventory table.
How was this patch tested?
Documentation only (RFC). The evidence in §2 was gathered live on the dual DGX Spark pair:
the devbox was started with
--gpu, a GPU-probe AppEnvironment was deployed and inspectedvia
kubectl exec(pod resources,/dev,/proc/driver/nvidia), and the devbox was torndown afterwards with the production DeepSeek V4 service untouched.
Docs link
rfc/system/20261022-devbox-dual-dgx-gpu.md